Laser ranging, tracking and designation using 3-d focal planes

ABSTRACT

By using 3-D focal plane arrays, the present invention tracks or locates small moving objects, or generates a 3-D frame of data with minimum laser energy and a minimum of mechanically moving parts. In another embodiment the invention is used to determine the direction of a laser designating a target with a minimum of moving parts. In another embodiment the invention is used as a 3-D movie/video camera. In yet another embodiment the device is used to provide data for autonomous navigation.

This application claims priority of U.S. Provisional Application60/748,690, filed Dec. 8, 2005.

FIELD OF THE INVENTION

This invention relates to laser pulse imaging of three dimensionalobjects. In particular it relates to embodiments of such technology fortarget acquisition and tracking, collision avoidance or navigation.

BACKGROUND OF THE INVENTION

The 3-D imaging technology disclosed in Stettner et al, U.S. Pat. Nos.5,446,529, 6,133,989 and 6,414,746 provides the use of a single pulse oflaser light to capture the information and content of a 2-D image alongwith a third dimension depth coordinate; thereby providing the 3-Dcoordinates of object points in its field of view. This has beenreferred to as flash 3-D imaging in analogy with ordinary digital 2-Dcameras using flash attachments for a self contained source of light. Aswith ordinary 2-D digital cameras, the light reflected from an object isfocused by a lens onto the focal plane of the camera, which is dividedinto an array of pixels called a focal plane array (FPA). In the case ofa 3-D camera these pixels are “smart” and can collect data from whichthe time of flight of the laser pulse to the object of interest can becalculated as well as data associated with the returning laser pulseshape and magnitude. Because of the similarity to radar imaging, thelaser light is also referred to as flash ladar. These flash 3-D camerasare an improvement upon designs in which one or more pixels is scannedover the field of view. They eliminate the need for a precisionmechanical scanner, which is costly and high maintenance; since thelocation of the pixels in the focal plane may be automaticallyregistered due to their permanent positions within an array.

An additional virtue of flash ladar is its ability to capture an entirescene in one exposure despite the rapid motion of parts or sections oftargets (such as the rotary blade of a helicopter) or rapid motion ofthe 3D camera's sensor platform. During the time it takes light to reachthe target and return to the sensor, mechanical systems typically do notmove fast enough to cause pixel blurring. A time sequence of 3-D flashladar frames comprises a 3-D movie/video of the scene.

BRIEF DESCRIPTION OF THE INVENTION

The present invention comprises apparatus and methods for flash 3Dsystems adapted for acquisition and tracking system suitable formilitary uses, collision avoidance systems suitable for vehiculartraffic, navigational systems and 3D motion picture systems, the latterbeing particularly suited for the development of video games.

The apparatus of the present invention comprises a laser source, a lens,a 3-D FPA and electronics to control the 3-D FPA, data derived from the3-D FPA and process the data. Each pixel in the 3-D FPA comprisesstructures to convert the light falling on it into an electronic signalwhose magnitude is sampled in time and stored in memory within thepixel. Each pixel also comprises a clock that reports the times at whichthe samples are taken. Different embodiments of the invention specifyvarious pulsed laser sources and optional 2-D sensors. The 2-D sensordata may be used to acquire targets or track targets in two dimensionsfor which the 3-D data may provide the target range to the pixel as athird dimension. The 2-D data may also be overlaid or textured on the3-D data to generate more realistic, higher resolution 3-D movies forhuman viewing.

It is an object of the present invention, in separate embodiments, tolocate or track small stationary or moving objects, to developfull-frame 3-D images, or to locate laser-designated targets with aminimum of laser energy, a minimum of mechanical motion or both.

It is another object of the present invention to provide a systememploying full-frame 3-D images in a collision avoidance system.

It is a further object of the present invention to provide a systememploying full-frame 3-D images in a navigation or a reconnaissancesystem.

It is a still further object of the present invention to provide asystem employing full-frame 3-D images in a 3-D movie/video.

It is still further the object of the present invention to provideapparatus and a method to identify the direction of a laser beamdesignating the platform on which the present invention is placed or tolocate the direction of a laser spot on a distant object.

Among the advantages of this device is the novel use of technologyhaving mechanical simplicity and low energy requirements for operation.The low weight and low volume for the device are additional advantagesand stem from the mechanical simplicity, the low energy requirements orboth. Other advantages and uses of the invention will be apparent tothose skilled in the art.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective drawing of the present invention intended toacquire and track moving objects or to locate objects in its field ofview.

FIG. 2 shows 3-D sensor intended to locate objects in its field of view,and locate the direction of a laser beam falling upon the sensor.

FIG. 3 shows the 3-D sensor electronics and focal plane array of thepresent invention.

FIG. 4 is a diagram of the 3-D focal plane array of the presentinvention.

FIG. 5 is a high sensitivity 3-D focal plane array of the presentinvention, an image tube.

FIG. 6 is a diagram of the unit cell circuitry on the 3-D focal planearray of the current invention.

FIG. 7 is an electronic block diagram of the current invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

When generating a flash image with a 3-D FPA each pixel must receiveenough laser energy so that a 3-D image can be generated. The fartherthe object of interest is from the camera the greater the output laserenergy required. Since for long-range systems, cost and power is relatedto the laser energy, reducing the requirement for laser power is abenefit. If the object of interest is only a few pixels in size, is faraway, and could be anywhere in the entire field of view of the 3-D FPA,it is most economical to narrow the transmitted laser beam to be only afew pixels in width and to scan the laser beam. Since scanning a laserbeam requires only a very small, inexpensive and extremely reliablegalvanometer scanner the present invention replaces the full flash ladarsystem by a cheaper, lower power and sometimes-lower weight and lowervolume Energy-Reduced Flash Ladar System (ERFLS).

In one embodiment a 2-D sensor acquires the object of interest andpasses the 2-D coordinates to the ERFLS for tracking. In anotherembodiment of the invention the laser beam is rapidly scanned to findthe object of interest within a time of concern, as in a collisionavoidance system where the object of interest may be a thin cable orwire. In still another embodiment of the invention, where the targetmotion is not too rapid, the entire image can be built up by scanningthe laser beam over the full field of view. In this latter embodiment,pixel registration is not a problem because all pixels are automaticallyregistered by being part of an array. Pixel registration, however, is aproblem in a system where the pixel is mechanically scanned to obtainthe image. A time sequence of ERFLS 3-D frames amounts to a 3-Dmovie/video of the scene which could be used in platform navigation.

There are situations in which one observer uses a pulsed laser beam todesignate a particular object of interest by projecting a laser “spot”on the object. In one manifestation of this application the laser spot'stwo-dimensional coordinates are determined by a separate distantobserver. Typically the distant observer scans a pixel or a small pixelarray over the full field of view of the system searching for the laserspot. As with other mechanical systems that scan a pixel, this system iscostly and high maintenance. In another embodiment of the presentinvention this scanned mechanical search system is replaced by a 3-DFPA, which takes in the full field of view of the pixel scanner. In amodification of this embodiment a 3-D FPA is used to identify thedirection of laser beam attempting to locate the platform on which the3-D FPA is mounted.

A preferred embodiment is described with reference to the figures wherelike numbers denote the same elements.

A preferred embodiment of the present invention, depicted in FIG. 1, isan acquisition and tracking system. It is comprised of a pulsed lightsource 1, compact scanning mirror system 2, including a scanning mirror5 a, a 3-D sensor 3, a 2-D acquisition sensor 4, and a control computer5 to store and process data. In other embodiments where the presentinvention is used as a collision avoidance or navigation system the 2-Dsensor may or may not be present. In addition where the objects in thefield of view of the 3-D sensor are poor reflectors of the light fromthe pulsed light source 1, the pulsed light source may generate enoughpower so that a cooler 6 is necessary to cool the pulsed light source 1.For those circumstances where a cooler is required the cooler 6 is partof the acquisition and tracking system. Preferably, the light source isa pulsed laser that creates a pulse with a pulse width of a fewnanoseconds. For use with near objects, the invention may only require apulse from a lower energy laser diode as the pulsed light source 1. Thepresent invention may also include a response system 99 to react to theobject being tracked. The light source may include a beam shapingelement 1 a, typically optics or a diffuser or both. The beam shapingelement may be incorporated into the scanning mirror 5 a as one or asystem of electronically controlled mirrors. These latter mirrors may bea Micro-Electro-Mechanical System (MEMS) fabricated with nanotechnology.

The 3-D sensor 3 is shown in FIG. 2 and is comprised of a lens system 7and a 3-D sensor housing 8. Inside the sensor housing 8 is the 3-D FPA10 (see FIG. 3) and 3-D sensor drive and output electronics 9.

FIG. 3 illustrates the 3-D sensor drive and output electronics 9, whichis typically comprised of a drive board 11 and the output and interfaceelectronics board 12. Typically the 3-D focal plane array (FPA) 10 iselectrically and mechanically attached to a drive board 11, whichprovides the clocks and biases to the Readout Integrated Circuit (ROIC)chip 14 (see FIG. 4) of the 3-D FPA. The drive board 11 may also providethe voltage biases required for the detector array 15 of the 3-D FPA 10.The output and interface electronics board is connected by means of acable 13 or a transmitter or a direct mechanical and electricalconnection to the control computer or computer board 5.

FIG. 4 shows the details of the FPA 10. The FPA 10 is comprised, in oneembodiment, of a light sensitive detector array 15 and an ROIC 14. FIG.4 indicates the detector array 15 and the ROIC chip 14 are divided intopixels 16 with a corresponding pixel 17 b on the detector array 15 and acorresponding pixel 17 a on the ROIC 14 that are electrically connectedby a metallic bump 18. In another monolithic embodiment the lightsensitive detector array 15 and ROIC array 14 chips are combined intoone chip. In one preferred embodiment, the light sensitive detectorarray 15 is a P-intrinsic-N (PIN) or a N-intrinsic-P photodiode or a APD(avalanche photodiode) fabricated on a single, low resistivitysemiconductor chip. In another embodiment, each light sensitive detectorpixel 17 b is incorporated into the readout pixel (or unit cell) 17 aitself in a way known by those familiar with the art. This monolithicdesign eliminates the complexity of two chips (14 and 15) hybridizedwith metallic bumps 18.

FIG. 5 shows an image tube embodiment of the FPA 10, in which the lightsensitive detector array 15 is a silicon detector array and the detectorarray 15 and ROIC array 14 are incorporated into an evacuated enclosurewhere photons P pass through a transparent window 95, interact with thephotocathode 98 of the image, producing photoelectrons e. Thephotoelectrons are accelerated into the detector array 15 by an electricfield E producing an amplified electrical signal by impact ionization inthe detector array. This amplified signal is then indistinguishable,when further processed, from a photon-produced electrical signal in thedetector array. In the case of an image tube, control, timing and biassignals flow through the header of the image tube 97, which iselectrically and mechanically attached, typically by pins 96 to thedrive and output electronics 9. In another image tube embodiment of theFPA 10, a microchannel plate is the electron amplification mechanism andis placed between the photocathode and ROIC 14 hybridized to detectorarray 15, however the detector array 15 is replaced by an anode array.The anode array simply collects the amplified electrical signals fromthe microchannel plate and passes them on through metal bumps 18 to theROIC 14. In another image tube embodiment of the 3-D FPA 10 the imagetube is placed in a magnetic field resulting in a digicon. Image tubes,headers, photocathodes, microchannel plates, anode arrays and digiconsare known to those skilled in the art of low-noise photon-signalamplification.

The electronic circuit 17 c on each ROIC unit cell (pixel) 17 a of theROIC chip 14 is diagrammed on FIG. 6. Each unit cell circuit 17 ccontains an input amplifier 27, a trigger circuit 21 delay circuit 22, acircular selector 19, a count accumulator 23, and several switches 28each connected to its own memory cell. An output amplifier 25 drives thedata in the memory cells 31 out of the unit cell to a chip outputamplifier which drives the data off the ROIC chip 14 and to the driveand output electronics 9. In alternate configurations of the ROIC unitcell 17 a the clock 29 may enter the count accumulator 23 and the countaccumulator 23 drive the circular selector 19. In an alternateconfiguration of the ROIC unit cell circuit 17 c the count accumulator23 and circular selector 19 may be combined. In still another alternatedesign ROIC unit cell 17 a which, does not capture the laser pulse shapethe circular selector 19, switches 28 and memory cells 31 may not bepresent.

In an alternate design of the ROIC chip 14 the unit cell data is notdriven off the chip but driven to an on-chip processor, which processesthe data. The processed data is then driven off the chip. The on-chipprocessor may be a parallel processor so that each row of unit cells hasits own processor. In alternate designs groups of rows may bemultiplexed to their own on-chip processor.

A block diagram of the system electronics is displayed on FIG. 7,composed of the drive and output electronics 9 and the master computer5. The focal plane array 10 receives its power and the bias voltagesneeded to run it from the power component 33. The embedded computer 34,typically a field programmable gate array (FPGA), generates the clocksignals for the focal plane array 10 and the memory 35. These includethe signals to initiate data acquisition, regulate raw data output fromthe focal plane array 10, and set options such as the clock frequency 29and delay time 22. Data from the focal plane array 10 is completely orpartially processed in the embedded computer 34 before being sent to thememory 35 and master computer 5 which may complete the processing, storethe results and display the acquired three dimensional image or activatethe response system 99. In another embodiment all the processing isperformed on the embedded computer 34, with the master computereliminated, and it activates the response system. The master computer 5may permit an operator to communicate with the drive and outputelectronics 9 and, indirectly, the focal plane array 10.

The Acquisition and Tracking System

The acquisition and tracking system 100 operates as follows: The 2-Dsensor 4, typically a visible or infrared sensor, acquires a target,which can be moving with respect to the acquisition and tracking system100. Typical targets are rockets, mortars, rocket propelled grenades,bullets and initial light emissions generated when these targets arefired. The 2-D camera transfers the two-dimensional x,y coordinateposition of the target either directly or by means of the computer 5 orembedded computer 34 to the two-axis scanning mirror system 2 whichrapidly orients its mirror 5 a in coordinates x and y so that a pulse oflight from the pulsed light source 1, through the beam shaping element 1a, reflected from the mirror 5 a will intersect the position of thetarget. The pulse of light emitted from the pulsed light source 1,traveling at the speed of light, intersects and reflects from thetarget. The reflected light is captured by the lens 7 of the 3-D sensor3 and focused on the focal plane array 10 attached to the drive andoutput electronics 9 of the 3-D sensor. The target typically subtendsone pixel 16 for acquisition and tracking but could subtend more thanone, particularly if the embodiment used is for collision avoidance,navigation, reconnaissance or 3-D movies/videos.

The light pulse reflected from the target generates an electricalcurrent in the detector pixel 17 b representative of the pulse shapegenerated by the pulse light source 1. This current passes through themetal bump 18 and into the corresponding ROIC unit cell 17 a. The targetmay modify the pulse shape but processing in the computer 5 or the driveand output electronics 9, embedded computer 34, will account for thepulse shape modification. The current from the detector pixel 17 b isinput to the ROIC unit cell circuitry 17 c in the ROIC unit cell 17 aand amplified by input amplifier 27 and then sampled and stored inmemory cells 31. Only three memory units are shown in FIG. 6 although inthe preferred embodiment many may exist. The sampling is accomplished bythe clock 18, circular selector 19 and switches 28. The circularselector has as many outputs as memory cells 17. At each write clockpulse, the circular selector shifts turning on a separate andindependent switch 28 which connects the input amplifier 27 output toone memory cell 31. After all memory cells have been filled the memorycells are overwritten as new data arrives. Consequently, the memorycells are always filled with the most recently sampled waveform data.The clock period is typically much shorter than the pulse width and sothe pulse shape is captured in the memory cells 31. If the inputamplifier 27 is a transimpedance amplifier the detector 17 b current istransformed to a voltage and the memory cells 31 sample this voltage. Ifthe input amplifier 27 is a current amplifier the detector 17 b currentis amplified and the memory cells 31 integrate this current. If thedetector 17 b signal is voltage the input amplifier 27 can be a voltageamplifier and the memory cells 31 sample this voltage. The circularselector 19 could be a simple sequential shift register but could bebased upon an algorithm and therefore not sequential.

In one preferred embodiment, sampling begins when the light pulse isemitted. The count accumulator 23 starts counting after one completecircular selector cycle and continues incrementing the count with eachsubsequent circular selector cycle. The input amplifier 27 output ismonitored by a trigger circuit 21, typically a Schmitt Trigger, whichoutputs a signal to a delay circuit 22 when the amplifier output 20reaches a pre-specified or drive and output electronics 9 adjustablemagnitude. After a preset, or drive and output electronics 9 adjustable,time delay, the delay circuit 22 stops the circular selector, whichterminates the sampling and counting. The count accumulator 23 countsthe number of times the circular selector has cycled or, in an alternatedesign, counts the number of clock pulses or equivalently the number oftimes the circular selector has shifted. In another preferredembodiment, sampling is begun in all pixels when a signal is input tothe ROIC 14 from the drive and output electronics 9. In another or thesame preferred embodiment, sampling is stopped when a signal is input tothe ROIC 14 from the drive and output electronics 9; this signal can beprogrammed to occur at a specific time. The count accumulator 23 couldbe a simple digital counter but could be based on a algorithm andtherefore not sequential. The algorithm would convert the output of thecount accumulator 23 to the actual count.

In alternate designs the clock pulse is first input to the countaccumulator, which drives the circular selector or the count accumulatorand circular selector are combined into one element. If the countaccumulator 23 drives the circular selector then the trigger circuit 21signal stops the count accumulator.

After the laser pulse has been sampled, the data in all the memory cellsis output by cycling through the circular selector at a lower frequencythan the typical write frequency (the frequency that the data wassampled at). Typically the input amplifier 27 is disconnected from theunit cell circuitry 17 c. The memory cell data is then driven from theunit cell to a chip output amplifier by the unit cell output amplifier25. The chip output amplifier drives the memory cell data to the outputand drive electronics 9, which processes the data and may transfer thedata to the processing and storage computer 5. Similarly, the unit cellcount accumulator 23 is output. This is typically a digital output whenthe clock is a series of pulses but could be analog if the clock is aramp voltage. The memory cell output is typically analog but could bedigital, particularly if the unit cell contained an analog to digitalconverter. Clock signals to output the data are generated in theembedded computer 34 of the drive and output electronics 9. Either theprocessing computer 5 or the embedded computer 34 computes the range tothe target, the third independent coordinate z.

In the acquisition and tracking system 100 embodiment of the currentinvention he x,y position of the target pixel (17 a and 17 b) on the 3-DFPA 10 corresponds to the x,y position of the target as determined bythe 2-D sensor 4, although typically the 2-D sensor has more pixels andits x,y position is more accurate. However, in another embodiment thedetector array 15 of the 3-D FPA 10 is responsive to the ambient lightreflected from the target or to the light emitted by the target and the2-D acquisition camera can be eliminated by modifying the ROIC chippixels 17 a so they can integrate target-reflected ambient light ortarget-produced-light as well as capture system 100 generated pulsedlight. In one embodiment eliminating the 2-D sensor the FIG. 6 unit cellcircuit is biased so that one or more of the memory 31 integrate lightat a low frequency typical of integrating sensors during acquisitionmode. Once the target is acquired in 2-D mode and its x,y coordinatesdetermined, the ROIC pixel circuit 17 c is then immediately re-biasedfor 3-D range acquisition to obtain the z target coordinate.

The Moving Target Acquisition and Tracking System

When operating as a moving target acquisition and tracking system 100the x,y coordinate of the moving target is tracked on the 2-D sensor andits corresponding z tracking coordinate is continually found using the3-D sensor. Thus the target track is developed in the embedded computer34 or processing computer 5, or an application specific integratedcircuit (ASIC) replacement of the drive and output electronics 9, andcan predict the origination point and termination point of thetrajectory using algorithms known to experts in the art. The computercan then command a response based on the trajectory computation, in thecase of a moving target, or when acquired in the case of a stationarytarget, from the response system 99. The response system may be acounter missile system, a counter shooter system, or vehicle controlsystem.

Typically the z position of the target in the moving target acquisitionand tracking system 100, and in all other embodiments which associate arange with a pixel 16, is determined both by the time at which the countaccumulator 23 stops and the pulse shape data acquired from the memorycells 31. Only the count accumulator 23 data is necessary if rangeprecision is to be determined within one or a few clock pulses. Rangeprecision of a fraction of a clock pulse can be achieved using the laserpulse shape data from the memory cells 31. Although many differentalgorithms can be used, the most accurate is a comparison of the actuallaser pulse shape with the pulse shape acquired from the memory cells31. Typically the arrival time of the laser pulse or the time of thepeak of the arriving laser pulse is estimated by a matched filteralgorithm or a least squares algorithm. Although these algorithms aretypically applied numerically they can be implemented in circuitry.

The Collision Avoidance System

When operating as a collision avoidance system 100, operation may besimilar to that described for the operation of the moving targetacquisition and tracking system 100: the object to be avoided is firstacquired using the 2-D sensor 4 operation, and the range to the objectof interest results from the acquisition of a laser pulse by the 3-Dsensor 3. However, typically the object of interest in thesecircumstances is defined by many pixels rather than one and may bescanned by a number of laser pulses, or just one, full field of view,laser pulse, to define the 3-D shape. The scanning pattern is typicallydefined or developed in the computer 5 but may be included in thefirmware of an embedded computer 34 located on the drive and outputelectronics 9 or an application specific integrated circuit (ASIC)replacement of the drive and output electronics. In addition thedivergence of the laser pulse may be large enough to illuminate manytarget pixels with a single pulse so that only few laser pulses, or onlyone, are necessary to obtain the full 3-D image. In an alternateembodiment the 3-D focal plane may be re-biased to function as a 2-Dacquisition sensor as well as a 3-D sensor and in still anotherembodiment the 3-D sensor may function as the acquisition sensor. If thesystem is designed so that only one laser pulse is necessary thescanning mirror system 2 may not be present and the laser 1 oriented sothat it illuminates the full field of view of the 3-D sensor 3 typicallyusing a beam expanding element 1 a.

In the embodiment of the collision avoidance or navigation system 100,where the 3-D sensor functions as an acquisition sensor, the 2-D sensor4 is not present and typically the pulse of light from the pulsed lightsource 1 has a beam divergence which illuminates many target pixels;1000 pixels is typical but all pixels may be illuminated by a singlepulse. 3-D image data is transferred to the computer 5 in real time foreach pulse of light and the 3-D image constructed from the scan. Thelaser beam scan will not generally be pixel by pixel but by groups ofpixels until all the pixels in the field of view of the 3-D sensor 3have been acquired. Then the scan will begin again. Typically the scanpattern will be located in the computer 5, in the embedded computer 34,or an ASIC substitute for the drive and output electronics 9. Theembedded computer 34 or the control computer 5 analyzes the 3-D data andactivates a response system 99 to avoid the collision or make navigationadjustments if necessary. Typical response systems include ailerons,steering brakes and accelerators. If power were not an issue the singlelaser flash could illuminate all the pixels in the field of view. If thesystem is designed so that only one laser pulse is necessary thescanning mirror system 2 may not be present and the laser 1 oriented sothat it illuminates the full field of view of the 3-D sensor 3 typicallyusing a beam expanding element 1 a.

The Reconnaissance or Movie Systems

In the embodiments of the reconnaissance (or 3-D movie/video camera)system 100, the 2-D sensor 4 may or may not be present and typically thepulse of light from the pulsed light source 1 has a beam divergencewhich illuminates many or all target pixels; 1000 pixels is typical. Abeam expanding element 1 a, typically a diffuser, may be employed tocontrol the divergence of the laser beam. 3-D image data is transferredto the embedded computer 34 or computer 5 in real time for each pulse oflight and the 3-D image constructed from the scan. Typically the laserpulse repetition frequency is 0.5-10,000 Hz and may depend upon the timedependence and desired time resolution of the scene being captured. Ifthe 2-D sensor is present the object of interest may be acquired withthe 2-D sensor and the 3-D sensor used for target identification (ID). Atypical system of this kind may be a rifle sight where the 2-D sensor isan IR sensor and the control computer 5, under human operator control,displays the 3-D data to the operator who makes the decision to respond.The rifle is the response system in this case. The operator could ofcourse be absent from the reconnaissance system 100 or at a remotelocation. The 2-D sensor data may be overlaid on the 3-D data to enhancethe target recognition. This overlay is sometimes referred to astexturing. The results of panning the 3-D reconnaissance camera over ascene with or without the 2-D overlay, at a frame rate of typically 30Hz, where typically one laser pulse covers the full field of view, is a3-D movie/video, hence 3-D movie/video camera. Overlaying lower spatial(x,y) resolution 3-D FPAs with higher spatial resolution 2-D FPAsincreases the realism or target ID probability. Lower spatial resolution3-D FPAs are desirable if the time resolution of the scene requires fullframes of data since the more 3-D pixels required at one time the higherthe laser energy required. Under those circumstances where full framesof 3-D data are required at high speed the mirror system 2 is not usedand the laser 1 is oriented to illuminate the full field of view.

In another preferred embodiment of the reconnaissance or 3-D movie/videocamera system 100 the beam shaping element 1 a can be varied so that theilluminated field of view of the 3-D sensor 3 can be varied from thefull field of view of the sensor to a few or a single pixel. Thereby thesystem 100 3-D sensor 3 use can be continuously varied from a full fieldof view 3-D imager to a range finder. The smaller the illumination fieldof view, or beam divergence, the longer the range that is possible witha given laser power; with a single pixel illumination field of view thegreatest range can be achieved.

Uses of the Movie System

One particular use of the movie system of the present invention is tocapture three dimensional motion of figures for incorporation into gamemachines. In particular the invention enables a single camera, withoutthe complexities of multiple lens systems employing parallax, to capturethe third dimension of characters participating in a sports event suchas a football game. From the data captured by the 3D camera persons ofskill in the programming art could enable the played back scene to berotated and approached from different geometrical points of view.

The Laser Designation System

In the laser designation embodiments 100 the invention is passive in itsacquisition mode; the laser 1, and mirror system 2 is not used. Neitheris the 2-D sensor 4 typically used. When used as a laser designator theobjective is to locate the x,y position of a target being designated bya pulsed laser beam from an independent, typically not collocated, lightsource. Light scattering off the designated target is focused by theoptics 7 onto the focal plane 10 of the 3-D sensor. One or at most a fewpixels 16 respond to the light. The position of these pixels on the 3-DFPA 10 is determined when the whole array 15 is read out into theembedded computer 34 or the control computer 5 and the computer alertsthe response system 99 to the x,y coordinates of the target. Typicallythe x,y position of the target is defined by the x,y position of thelaser spot on the 3-D FPA 10 and the focal length of the receive optics7. The computer may identify the designation laser using pulserepetition frequency or pulse shape matching algorithms. Under somecircumstances both pulse repetition frequency and pulse shape may beused to identify the designation laser. A typical response system is amissile system.

When the laser designator 100 is being illuminated directly by a laserbeam, light is focused by the optics 7 onto the focal plane 10 of the3-D sensor. One or at most a few pixels 16 respond to the light. Theposition of these pixels on the 3-D FPA 10 is determined when the wholearray 15 is read out into the embedded computer 34 or control computer 5and the computer alerts the response system 99 to the x,y coordinates ofthe laser beam. Typically the x,y position of the target is defined bythe x,y position of the laser spot on the 3-D FPA 10 and the focallength of the receive optics 7. A typical response system is thereconnaissance embodiment 100 of the present invention to identify thelaser source. When the image tube FPA 10 is used with the 3-D sensor 3in any of the embodiments, laser light passes through the window 95 andinteracts with the photocathode 98 producing an electron, e. Theelectron is accelerated by the internal electric field E to a relativelyhigh energy, typically measured in KeV, until impact of the electronwith the detector array 15 causes the generation of many electron-holepairs. Either the electrons or the holes are swept into the ROIC 14,though the metal bump 18, by an electric field internal to the detector,and the process is then the same as for the solid state array depictedin FIG. 4.

1-58. (canceled)
 59. A device for obtaining 3-D data sufficient togeometrically define the surface of a moving or stationary objectcomprising: a light source providing successive light pulses, a beamshaper directing the light pulses into a vacuum or a light transmittingmedium between the object and the pulsed light source and illuminatingat least a portion of the surface of the object, a sensor that detectslight pulses that are returned from the medium and from one or moreportions of the object and which provides electrical signals indicativeof the range of one or more portions of the object, said sensorcomprising a focal plane array comprising pixel elements for providingsaid electrical signals, said pixel elements of the focal plane arraycomprising unit cell electronics and light detectors, light collectingoptics operable during intervals between the transmission of successivepulses from said light source to collect light received at pixelelements on said focal plane array, said detectors creating primaryelectrical signals corresponding to the light received at said pixelelement, said unit cell electronics adapted to preprocess the primaryelectrical signals and to output electrical signals, a processor of saidelectrical signals from said sensor into 3-D range data corresponding toeach pixel, and drive and output electronics providing control signalsto said focal plane array and providing the 3-D range data to a 3-D dataprocessor or to a storage device for storing the 3-D range data.
 60. Thedevice of claim 59 including a 2-D digital camera, which obtains 2-Ddata of the moving or stationary object in conjunction with said 3-Drange data.
 61. The device of claim 60 said sensor comprises a 2-Ddigital camera that can be electronically switched between the 3-D datacollection mode and the 2-D data collection mode.
 62. The device ofclaim 60 wherein the said processor of electrical signals includes aprocessor to overlay, superpose or texture 2-D digital data on the 3-Ddata.
 63. The device of claim 59 wherein the beam shaper includes a oneor multiple axis scanning mirror to direct said light.
 64. The device ofclaim 59 wherein the light shaper includes an electromechanical systemwith one or a plurality of independently controlled mirrors to directsaid light.
 65. The device of claim 59 wherein the light shaper includesone or more selectable diffusers subtending particular transmitted-lightfields of view.
 66. The device of claim 59 wherein the light shaperincludes variable field of view optics so that the device can operateboth as range finder and 3-D imager
 67. The device of claim 59 whereinthe processor for electrical signals from said 3-D sensor into 3-D datacorresponding to each pixel is included on the 3-D focal plane array.68. The device of claim 59 wherein the processor for electrical signalsfrom said 3-D sensor into 3-D data corresponding to each pixel isincorporated in a Field Programmable Gate Array (FPGA).
 69. The deviceof claim 59, wherein the said processor of electrical signals includes aprocessor for developing the 3-D data into a 3-D image.
 70. The deviceof claim 59 wherein the said processor of electrical signals includes aprocessor for converting said 3-D data, and 2-D, if available andnecessary, into collision avoidance or navigation data.
 71. The deviceof claim 59 wherein the said processor of electrical signals includes aprocessor to convert said 3-D data, and 2-D data, if available andnecessary, into 3-D movie or 3-D video data.
 72. The Device of claim 71wherein the 3-D movie is of a sports event.
 73. The device of claim 59wherein the said processor of electrical signals includes a processor toconvert said 3-D data to projectile trajectory data.
 74. A method fortracking, investigating, filming, navigating through or avoidingcollision with one or more stationary or moving objects comprisingtransferring the two-dimensional coordinate positions of the object orobjects to a controllable mirror as many times as necessary to obtaintime dependent data frames or object trajectories, reflecting a pulse oflight from a pulsed light source through a beam shaping element and thesaid controllable mirror to intersect and reflect from the object orobjects, for each of said transferred two-dimensional coordinate,focusing the light reflected from the target onto a 3-D focal planearray attached to the drive and output electronics of a 3-D sensor,sampling the electrical signals generated in the 3-D focal plane arraypixels and storing the 3-D data in memory cells determining objectcoordinates and/or velocities from the stored 3-D data.
 75. The methodfor tracking, investigating, filming, navigating through or avoidingcollision with one or more stationary or moving objects of claim 74where the said objects are acquired and the said two-dimensionalcoordinate positions generated by using a 2-D sensor
 76. The method fortracking, investigating, filming, navigating through or avoidingcollision with one or more stationary or moving objects of claim 74wherein the object comprises an initial light emission typicallygenerated during firing of a projectile.
 77. The method for tracking,investigating, filming, navigating through or avoiding collision withone or more stationary or moving objects of claim 74 wherein the objectcoordinates and/or velocities are transferred as 3-D images to a digitalrecording media.
 78. A method for tracking, investigating, filming,navigating through or avoiding collision with one or more stationary ormoving objects comprising transmitting a sequence of light pulses from apulsed light source to intersect and reflect from the said objects, saidpulsed light source transmitter illuminating the field of view of a 3-Dsensor and each pulse associated with a frame of data focusing reflectedlight from the said one or more stationary or moving objects onto afocal plane array attached to the drive and output electronics of a 3-Dsensor, sampling the electrical signals generated in pixels of the focalplane array and storing the 3-D data in memory cells, determining objectcoordinates and/or velocities from the stored 3-D data output.
 79. Themethod for tracking, investigating, filming, navigating through oravoiding collision with one or more stationary or moving objects ofclaim 78 wherein the transmitter field of view can be varied allowinggreater ranges, for the same pulse energy with smaller fields of view.80. The method for tracking, investigating, filming, navigating throughor avoiding collision with one or more stationary or moving objects ofclaim 78 wherein the object coordinates and/or velocities aretransferred as 3-D images to a digital recording media.
 81. The methodfor tracking, investigating, filming, navigating through or avoidingcollision with one or more stationary or moving objects of claim 80wherein 2-D sensor data is textured or overlaid on the 3-D imagery. 82.The method for tracking, investigating, filming, navigating through oravoiding collision with one or more stationary or moving objects ofclaim 80 wherein all or part the 3-D imagery is a broadcast to remotereceivers which can display the imagery.
 83. The method for tracking,investigating, filming, navigating through or avoiding collision withone or more stationary or moving objects of claim 80 wherein all or partof the 3-D imagery is used as material in a video game.
 84. The methodfor tracking, investigating, filming, navigating through or avoidingcollision with one or more stationary or moving objects of claim 80wherein all or part of the 3-D imagery is used to provide feedback indeveloping athletic performance.
 85. A method for obtaining 3-D datasufficient to geometrically define the surface of a moving or stationaryobject comprising: providing successive light pulses from a lightsource, directing the light pulses into a vacuum or light transmittingmedium between the object and the pulsed light source and illuminatingat least a portion of the surface of the object, detecting light pulsesthat are returned from the medium and from one or more portions of theobject and providing electrical signals indicative of the range of oneor more portions of the object, operating light collecting optics duringor between intervals of the transmission of successive pulses from saidlight source and collecting light received at pixel elements on saidfocal plane array, creating primary electrical signals corresponding tothe light received at the pixel element, preprocessing the primaryelectrical signals and outputting electrical signals representative of3-D range data corresponding to each pixel.
 86. The method of claim 85including obtaining 2-D data of the moving or stationary object togetherwith said 3-D range data.
 87. The method of claim 86 further comprisingswitching to a 2-D digital camera where the 3-D camera can beelectronically switched between 2-D and 3-D camera modes.
 88. The methodof claim 85 further comprising overlaying, superposing or texturing 2-Ddigital data on the 3-D data.
 89. The method of claim 85 furthercomprising directing said light via a one or multiple axis scanningmirror.
 90. The method of claim 85 further comprising directing saidlight via an electromechanical system with one or a plurality ofindependently controlled mirrors.
 91. The method of claim 85 furthercomprising converting said 3-D data, and 2-D data, if available andnecessary, into collision avoidance or navigation data.
 92. The methodof claim 85 further comprising converting said 3-D data, and 2-D data,if available and necessary, into 3-D movie or 3-D video data.
 93. Themethod of claim 85 further comprising converting said 3-D data toprojectile trajectory data.
 94. A device to acquire and track a movingobject having a surface comprising: a light source providing successivelight pulses, a beam shaper directing the light pulses into a vacuum ora light transmitting medium between the object and the pulsed lightsource and illuminating at least a portion of the surface of the object,a sensor that detects light pulses that are returned from the medium andfrom one or more portions of the object and which provides electricalsignals indicative of the range of one or more portions of the object,said sensor comprising a focal plane array comprising pixel elements forproviding said electrical signals, said pixel elements of the focalplane array comprising unit cell electronics and light detectors, lightcollecting optics operable during intervals between the transmission ofsuccessive pulses from said light source to collect light received atpixel elements on said focal plane array, said detectors creatingprimary electrical signals corresponding to the light received at saidpixel element, said unit cell electronics adapted to preprocess theprimary electrical signals and to output electrical signals, a processorof said electrical signals from said sensor into 3-D range datacorresponding to each pixel, and drive and output electronics providingcontrol signals to said focal plane array and providing the 3-D rangedata to a 3-D data processor or to a storage device for storing the 3-Drange data, said data being sufficient to compute end points of themoving object trajectory.
 95. The device to acquire and track of claim94, further comprising a response contrivance, which takes a physical orelectronic action in response to said data.
 96. The device to acquireand track of claim 95, wherein the response contrivance is projectilelauncher whose outgoing projectile is designed to intersect the computedtrajectory of the moving object.
 97. The device to acquire and track ofclaim 96, wherein the moving object is selected from the groupconsisting of a rocket propelled grenade, a bullet, or a heat seekingmissile.
 98. The device to acquire and track of claim 94 furthercomprising a write clock, circular selector and switches for samplingand storing in ROIC (readout integrated circuit) memory.
 99. A methodfor acquiring and/or tracking a target comprising acquiring the targetwith a 2-D sensor, transferring a two-dimensional coordinate position ofthe target to a two-axis scanning mirror system, reflecting a pulse oflight from a pulsed light source through a beam shaping element and themirror to intersect and reflect from the target, focusing the lightreflected from the target onto a focal plane array attached to the driveand output electronics of a 3-D sensor, inputting the electrical currentto an ROIC (readout integrated circuit) unit cell, sampling and storingthe electrical current signals in ROIC memory cells determining targetcoordinates and/or velocities from the stored ROIC output.
 100. Themethod for acquiring and/or tracking a target of claim 99 wherein ateach write clock pulse, the circular selector shifts turning on aseparate and independent switch which connects an input amplifier outputto one memory cell.
 101. The method for acquiring and/or tracking atarget of claim 99 wherein the input amplifier is a transimpedanceamplifier and a detector current is transformed to a voltage and thememory cells sample this voltage.
 102. The method for acquiring and/ortracking a target of claim 100 wherein the input amplifier is a currentamplifier and the detector current is amplified and the memory cellsintegrate this current.
 103. The method for acquiring and/or tracking atarget of claim 101 wherein the input amplifier is a voltage amplifierand the memory cells sample the voltage from the voltage amplifier. 104.The method for acquiring and/or tracking a target of claim 99 whereinsampling begins when a light pulse is emitted.
 105. The method foracquiring and/or tracking a target of claim 99 wherein sampling beginswhen a signal is input to the ROIC from drive and output electronics106. A device to acquire and track a light emission during missilefiring or laser designation comprising: a sensor that detects said lightemissions, said sensor comprising a focal plane array comprising pixelelements for providing said electrical signals, said pixel elements ofthe focal plane array comprising unit cell electronics and lightdetectors, light collecting optics to collect light received at pixelelements on said focal plane array, said detectors creating primaryelectrical signals corresponding to the light received at said pixelelement, said unit cell electronics adapted to preprocess the primaryelectrical signals and to output electrical signals, a processor of saidelectrical signals from said sensor into 3-D range data corresponding toeach pixel, and drive and output electronics providing control signalsto said focal plane array and providing the 3-D range data to a 3-D dataprocessor or to a storage device for storing the 3-D range data, saiddata being sufficient to compute position information concerning saidfiring or laser designation.
 107. A method for responding to a movingtarget comprising tracking the target on a 2-D sensor, emitting a pulseand receiving its reflection from the target, collecting the reflectionin an optical system and directing the collected reflection to a focalplane of a 3D sensor comprising pixels having memory cells, determiningby pulse shape data and timing data acquired from the memory cells acorresponding third dimension tracking coordinate developing a targettrack, predicting the origination point and termination point of thetrajectory, commanding a response based on the target track.
 108. Themethod of claim 107, wherein the 2D and 3D sensors are two states ofsingle sensor.
 109. A method for avoiding collision with a targetcomprising acquiring the target with a 2-D sensor, transferring atwo-dimensional coordinate position of the target to a two-axis scanningmirror system, reflecting multiple pulses of light from a divergentpulsed light source through a beam shaping element and reflected fromthe mirror to intersect and reflect from the target, focusing the lightreflected from the target onto a focal plane array attached to the driveand output electronics of a 3-D sensor, converting the light toelectrical currents, inputting the electrical currents to ROIC (readoutintegrated circuit) unit cells determining target parameters from thestate of the ROIC cells, and taking action based on the targetparameters to avoid collision.
 110. A 3-D sensor that detects reflectedlight pulses that are returned through a vacuum or light conductingmedium from one or more portions of an object and which provideselectrical signals indicative of the range of one or more portions ofthe object, said sensor comprising a focal plane array comprising pixelelements for providing said electrical signals, said pixel elements ofthe focal plane array comprising unit cell electronics and lightdetectors, said detectors creating primary electrical signalscorresponding to the light received at said pixel element, said unitcell electronics adapted to preprocess the primary electrical signalsand to output electrical signals, a processor of said electrical signalsfrom said sensor into 3-D range data corresponding to each pixel, anddrive and output electronics providing control signals to said focalplane array and providing the 3-D range data to a 3-D data processor orto a storage device for storing the 3-D range data
 111. The focal planearray of claim 110 wherein the focal plane array is incorporated into animage tube.
 112. The focal plane of claim 110, wherein each of said unitcells is comprised of input amplifiers of the primary electrical signalstransferred from the corresponding detector samplers of the saidelectrical signals at one or a plurality of time intervals one or morestorage means for storing said samples, a trigger circuit to stop aclock, electronics to record the time at which the said electricalsignals are stored or the clock is stopped or both the electricalsignals and the time at which the clock is stopped.
 113. The device ofclaim 110 wherein the focal plane includes one or more selectable clocksthat output a sequence of electrical pulses at regular time intervals.114. The device of claim 110 wherein the focal plane includes one ormore selectable clocks that output a ramp voltage.
 115. The device ofclaim 110 wherein the said electrical signals are representative of thepulse shape and amplitude of the said reflected light.
 116. The deviceof claim 112 wherein the means for input amplification of primaryelectrical signals is a transimpedance amplifier.
 117. The device ofclaim 112 wherein the means for input amplification of primaryelectrical signals is a current amplifier.
 118. The device of claim 112wherein the means for input amplification of primary electrical signalsis a voltage amplifier.
 119. The device of claim 112 wherein the meansto sample the electrical signal is voltage sampling and the sampling istimed by a clock consisting of a sequence of electrical pulses atregular time intervals
 120. The device of claim 112 wherein the means tosample the electrical signal is charge integration and the sampling istimed by a clock consisting of a sequence of electrical pulses atregular time intervals.
 121. The device of claim 112 wherein the meansto sample the electrical signal is voltage sampler timed by an analogcircuit.
 122. The device of claim 112 wherein the means to sample theelectrical signal is current integrator timed by an analog circuit. 123.The device of claim 112 wherein each of said storage means is acapacitor.
 124. The device of claim 112 wherein the trigger circuit is aSchmidt Trigger.
 125. The device of claim 112 wherein the electronics torecord the time at which the electrical signals are stored is a countaccumulator wherein the count is based upon an algorithm or the means torecord time is a digital counter.
 126. The device of claim 112 whereinthe means to record the time at which the electrical signals are storedis a capacitor.
 127. The device of claim 112 wherein the means to samplesaid electronic signals at one or a plurality of time intervals and oneor plurality of storage means for storing said samples is absent. 128.The device of claim 112 wherein the processor to process electricalsignals from said sensor into range data corresponding to each pixel isincorporated in processing circuitry on the focal plane array.
 129. Thedevice of claim 112 wherein the processor to process the electricalsignals from said sensor into range data corresponding to each pixel isincorporated as firmware in a Field Programmable Gate Array (FPGA),wherein the FPGA is part of the drive and output electronics.